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Abstract — Given a network infrastructure (e.g., data-center or 
on-cIiip-networl() and a distribution on tlie source-destination 
requests, tlie expected patli (route) length is an important 
measure for the performance, efficiency and power consumption 
of the network. In this work we initiate a study on self-adjusting 
networks: networks that use local-distributed mechanisms to 
adjust the position of the nodes (e.g., virtual machines) in the 
network to best fit the route requests distribution. Finding the 
optimal placement of nodes is defined as the minimum expected 
path length (MEPL) problem. This is a generalization of the 
minimum linear arrangement (MLA) problem where the network 
infrastructure is a line and the computation is done centrally. 
In contrast to previous work, we study the distributed version 
and give efficient and simple approximation algorithms for 
interesting and practically relevant special cases of the problem. 
In particular, we consider grid networks in which the distribution 
of requests is a symmetric product distribution. In this setting, we 
show that a simple greedy policy of position switching between 
neighboring nodes to locally minimize an objective function, 
achieves good approximation ratios. We are able to prove this 
result using the useful notions of expected rank of the distribution 
and the expected distance to the center of the graph. 

1. Introduction 

In the last decade we have witnessed two new major and 
related phenomena in distributed computing. The first is the 
emerge of huge data centers and warehouse-scale computers. 
The second phenomenon is the decentralization and paral- 
lelism of workload in single multi-core computers. In both 
cases (but on different scale) the system is a network of 
computing primitives that share global computational goals. 
In data centers networks, as well as in modem multiproces- 
sor computers, multiple processes run in parallel to execute 
common tasks so, in many cases, these processes need to 
communicate with each other to work on their shared tasks. 

Reducing energy waste, and in particular the power con- 
sumption of computing is one of the major challenges of the 
21st century. Both data centers and single computers are no 
exception, and constantly increasing their energy and power 
usage. For example, the total cost of power consumption of 
data centers in the USA alone is estimated to be 50 billion 
dollars [1]. Moreover, the energy consumed by data centers is 
estimated to double every five years [2]. The focus of this work 
is to improve upon the energy that is consumed by routing in 
such systems. It is estimated that in data centers the energy 
consumed by routing is about 20%-30% of the total energy [3]. 



Routing in network-on-chip (NoC) consumes even up to 50% 
of the total energy [4]. These numbers pose our community 
both an opportunity and a challenge. The opportunity is to gain 
significant energy savings for these systems; the challenge is 
to design and implement clever and simple algorithms that can 
improve routing efficiency. 

Another common property of these systems is that they all 
operate in a fixed network infrastructure. This means that we 
cannot change the structure of the network by, for example, 
rewiring links. But instead, what we can do, is to move the 
locations of processes (e.g., virtual machines) between the 
different computers (or CPUs). In this paper, we formulate 
the problem of saving energy on a fixed infrastructure network 
using migration of processes. The basic idea is that the energy 
cost of routing in a network is proportional to the length 
of the routes which suggests the following: If we can make 
the routes lengths (or the expected route length) shorter, 
then we can save energy. We devise local and distributed 
algorithms that (re-)place processes in the network to reduce 
the expected path length. This can be achieved, for example, by 
Software Defined Networking (SDN) [5] - the concept, which 
provides, among others, much better control over the network 
functionality. In SDN, a software management platform may 
support an abstraction for moving a selected process from one 
physical machine to another. Recently, this approach became 
practical, when Google announced [6] the implementation of 
OpenFlow [7] in its own backbone. 

The problem of minimizing the total energy consumed by 
routing is dependent on two major properties of the system: 
(i) the infrastructure (topology) of the communication network 
and (u) the statistical pattern of route requests between sources 
and destinations. We first show that even in a very simple 
pattern such as every node has an activity level and the 
probability to send or receive a message is proportional to 
its level, the problem is NP-complete on general network 
topologies. Secondly, even when the network is "simpler" or 
regular, hke a grid network, the problem can still be hard 
if the request distribution is "complex" in some sense. With 
this in mind we turn to analyze approximation algorithms 
for the setting where both the topology and the requests 
have nice properties. Our routing and activity distributions 
are partially justified from real data [8], [9]. We concentrate 
on local and distributed algorithms, namely, processes can be 



exchanged (i.e., relocated) only between nearby nodes without 
any centraUzed coordination. 

A. Overview of our results 

First, we formulate the discussed problem as the minimum 
expected path length (MEPL) problem, that is, given a network 
infrastructure and a distribution of requests, minimizing route 
costs by finding an optimal placement for processes in the 
network. When the network is a line, MEPL is identical to 
the minimum linear arrangement (MLA) [10] which is known 
to be NP-complete. In this work we consider d-dimensional 
grid networks, d>l, and requests that comes independently 
from a symmetric product distribution where the frequency of 
a route request (w, v) is a multiphcation of the activity levels 
of both u and v. In contrast to previous works, our goal is to 
design simple distributed algorithms for these more reaUstic 
settings. 

We first show that MEPL is NP-complete if (i): we only 
assume that the network is a 2-dimensional grid, and (ii): we 
only assimie that the requests come from a symmetric product 
distribution. But, somewhat surprisingly, if both conditions 
hold, we are able to present a simple, local, distributed algo- 
rithm that achieves good approximation to the optimal solution 
for the MEPL problem. Our algorithm is self-adjustable in the 
sense that nodes switch processes based on the continuously 
observed sequence of route requests each node is involved in. 
This approach was inspired and bears some similarity to self- 
adjusting data structures like splay trees [11]. In particuliir we 
are able to show (informal): 

Theorem. For a d-dimensional grid network and a symmetric 
product distribution of requests there is a simple distributed 
algorithm, which defines a local switching policy between a 
process and its neighbors that achieves a constant approxima- 
tion to the minimum expected path length (MEPL) problem. 

Interestingly, we prove this theorem using a measure called 
expected rank which is related to the uncertainty of a random 
variable in a similar manner as entropy is. 

We then tum to more complex distributions of requests and 
discuss requests that are clustered into disjoint groups. While 
for this setting few extremely unstable bad local minima can 
exist we present promising simulation results. In particular 
we show that for the 2-dimensional grid that starting fi-om a 
random and thus almost worst case initial state of processes 
locations in the network our local algorithms converge to an 
almost optimal local minimum. 

Organization: In Section II we discuss related work and 
somewhat similar approaches. Section III introduces the for- 
mal problem and definitions. The hardness of MEPL is proved 
in Section IV and then in Section V we prove our main result, 
a constant factor approximation in d-dimensional meshes with 
product route distributions. In Section VI we discuss a more 
complex setting: clustered requests; and we end the paper with 
a short conclusion in Section VII. 



II. Related Work 

Energy saving along with green computing is an active 
topic of research in the recent years. In a recent paper [12] 
Lis et al. study memory architectures of microprocessors. 
The authors suggest that processes will migrate to a location 
that is closer to the data instead of what is common in 
today architectures, i.e., coping the data to be closer to the 
process. The logic behind this idea is that programs are much 
smaller than their data. We take this idea one step further by 
reducing the communication distance between two processes. 
Improving the energy efficiency of routing in networks was 
also considered. Batista et al. [13] used traffic engineering on 
grids to self-adjust to routing requests. In [3] and [14] different 
authors considered data centers and tried to save energy by 
powers down routers and links when demand in the network is 
low. Other self-adjusting routing scheme were considered, for 
example in scale-free networks to overcome congestion [15], 
[16]. 

The most related areas of research to our study are graph 
arrangement, embedding and labeling problems [17], [18]. 
The basic question there is to embed a guest graph G into 
a host graph H in order to minimize some objective function 
like the bandwidth or the cutwidth; we relate our study to this 
settings in the model section. In particular, some VLSI design 
problems where considered on a two dimensional grids. [19], 
[20]. There are two significant differences here: first, we 
consider distributions on the route requests which restrict 
our guest graphs and second, and more importantly, we are 
interested in distributed, self-adjusting algorithms to solve the 
problem and not a centralized solution. 

As described in the introduction the self-adjusting nature 
of our solution was inspired by self-adjusting data structures 
like splay trees [11] which adjust their structure according 
to requests made to the data structure in such a way that 
the amortized cost matches the cost of the optimal (static) 
solution. The local greedy switch strategy we use is related to 
physics and natural dynamics which indirectly try to minimize 
energy. Using this analogy for optimization purposes has a 
long history. E.g., simulated annealing [21] can be seen as 
simulating physics while cooling the temperature, i.e., the 
local moves selected shift over time more and more bias 
from mostly random behavior to greedy energy minimization. 
Here we only look at greedy steps. In a networking context 
similar approaches were used for load balancing via diffusive 
paradigms [22] and for routing via gradient mechanisms [23] . 

Another very related research is about self-stabihzing 
graphs [24], [25]. The goal there is also to maintain some 
objective using local edge exchanges, mostly in an overlay 
network. In a similar manner we would like to extend the 
current work to solve MEPL on overlay and peer-to-peer 
networks, using edge rewiring as well. 

111. Model and Problem Definition 

We model the communication network by an undirected, 

unweighted and connected host graph H. Given a graph, its 
vertex set is denoted V{H) and its edge set by E{H). We 



denote the number of nodes with n = \ V{H)\ and the number 
of edges with m = \E{H)\. Let djj (•) be the distance function 
between nodes in H, i.e., for two nodes u,v G H we define 
(Ih {u, v) to be the number of edges of a shortest path between 
u and V in H. 

We assume that the network serves route requests drawn 
independently from an arbitrary distribution V and messages 
are routed along the shortest paths in H. Alternately, we 
represent the distribution V as a weighted directed guest graph 
G where |1^(G)| = n. For a directed edge {u,v) € E{G) let 
the weight of the edge p(u, v) denote the probabihty of a route 
request for a message from node u to v. 

Given a network infrastructure host graph H and a distri- 
bution on the route requests represented by a guest graph G, a 
placement (or labeling [17]) function is a bijective' function 
(fi : V{G) V{H) which determines the locations of nodes 
of G (processes) in the network H (hosts). Given G, H and 
a placement function (p the expected path length of route 
requests is defined as: 

EPL{G,H,^)= Yl p{u,v)-dH{v^{u),ip{v)) 

uveE{G) 

When H and G are clear from the context we may write 
just EPL{(f). Note a special case of this definition, when P 
is the uniform distribution: this gives the average path length 
in the network which is often used in the literature instead of 
the diameter, for example to show that a network is a small 
world network [26]. 

For H and V we would like to find an optimal placement 
of the nodes in the network to minimize the expected path 
length. Formally: 

Definition 1 {Minimum Expected Path Length problem). 

Given a host graph H and a probability distribution repre- 
sented by a guest graph G, find a placement function that 
minimize the expected path length: 

MEPL = minEPL(G, H, ip) (1) 

As mentioned earlier, this problem is motivated by the 
network serving point-to-point routing requests that are in- 
dependently sampled from a distribution V. If we assume that 
the cost for a request u, v is d{u, v) then the MEPL problem 
tries to minimize the expected cost of a route. Note that this is 
also equivalent to minimizing the expected usage of all links 
or minimizing the expected total number of transmissions - all 
important metrics in terms of energy saving and efficiency. 

In this work, we mostly consider local and distributed 
switching rules to find a good placement: rules where a 
node is only allowed to switch places with nodes that are 
in its neighborhood (i.e., close to it). The goal is that after a 
sequence of local switches the network will reach its minimum 
expected path length and solve the MEPL problem. On the 
one hand, our results from Section IV will show that this is 

' In this work we consider the basic case where every host machine can run 
at most one virtual machine. 



not possible (efficiently) in a general setting even with global 
knowledge and non-local switches. Throughout the paper we 
thus consider at times simpler forms of networks and requests 
distributions, i.e., grid networks and the symmetric product 

distributions: 

Definition 2 (d-dimensional grid networks). A mesh network 
of size n = k'^ with nodes embedded on all locations [kY 
where \k\ is the set of integers l,2,...,fc. Each node is 
connected to all the nodes at £i-distance one from it, i.e., each 
node has at most two neighbors in each of the d dimensions. 

Definition 3 {symmetric product distribution). In symmetric 
product distribution, each node has a level of activity and 
the more two nodes u and v are active the more likely that 
the route {u, v} gets requested. More precisely, we scale the 
activity levels of the nodes such that they form a distribution 
with an activity level p(w) for each node u and assume that the 
request distribution is induced by the product of the activity 
levels, i.e., p(u, v) = p{u) ■ p{v). 

IV. Hardness of MEPL 

In this section we show that solving the general MEPL 
problem is hard. Indeed, we prove two results that demonstrate 
how the hardness of the problem can come form either an 
involved network topology G or the structure of the routing 
request distribution P. This serves also as an additional 
motivation why in the rest of the paper we turn to graphs 
and distributions with more realistic structure. 

For both our examples it suffices to use probabihty distri- 
butions that only have one non-zero probability value. In our 
first statement, we show that even if we restrict ourselves to 
symmetric product distributions, the MEPL problem is hard 
on general networks: 

Lenuna 1. Given a host graph H and a symmetric product 
distribution P, it is NP-complete to decide whether the MEPL 
is smaller than a given value. 

Proof: We describe a reduction from the fc-CLIQUE prob- 
lem. In the fc-CLIQUE problem, one is given a graph H' and 
has to decide whether H' contains a fc-clique, that is, whether 
H' contains a complete graph on k nodes as a subgraph. 
This is one of Karp's 21 NP-complete problems [27]. For 
the reduction we take the graph H' as the network's host 
graph H. As a request distribution we use a symmetric product 
probability distribution that puts probability weight on 

each of the pairs V' x V' formed by a subset of the nodes of 
size k and zero probability on any other pair. If H contains 
a fc-clique, then the unique optimal solution to MEPL with 
value 1 — 1/fc will be obtained if all k nodes are placed in 
this clique. If H does not contain a fc-chque the there will be 
at least one request pair u,v gV xV that is at least two far 
apart and the total cost will be at least 1 — 1/fc + l/fc^. Thus, 
deciding whether the MEPL is smaller than 1 — l/fc + 1/k^ 
is equivalent to deciding whether H' has a fc-CLIQUE and 
thus, NP-hard. Lastly, it is easy to see that deciding whether 
the MEPL is smaller than a given value problem can be easily 



achieved in NP by guessing and then verifying a solution with 
smaller value. ■ 
This lemma shows that solving the MEPL problem for 
general network topologies is hard. Next, we show that even 
if we restrict the graph to be nice, e.g., 2-dimensional grid, a 
lack of structure in the probability distribution can make the 
MEPL problem hard, too: 

Lemma 2. Given a probability distribution V, it is NP- 

complete to decide whether the MEPL is smaller than a given 
value on a 2-dimensional grid network. 

Proof: We describe a reduction from the problem of 
embedding a tree in a 2-dunensional grid which was shown 
to be NP-complete by Bhatt and Cosmadakis [19]. More 
precisely, it is NP-hard to decide whether a given tree T 
(with maximum degree 4) is a subgraph of the grid. Given 
an instance of this problem in form of a tree T we construct a 
hard MEPL instance as follows: We take the two-dimensional 
grid \k]^ as a topology where k equals the number of nodes 
in T. As a request distribution we take a subset of k nodes 
to correspond to nodes in T and put a probability mass of 
l/(fc — 1) on each pair of nodes that corresponds to two 
neighbors in the tree T; all other fc^ — (fc — 1) node pairs 
have a probability of zero. The MEPL for such an instance is 
1 if and only if the tree can be embedded in the grid. If this 
is not the case, then at least one request pair will be separated 
by a path of length at least two increasing the average to 
at least 1 + l/{k - 1). Thus deciding whether the MEPL is 
smaller than l + l/(fc— 1) is equivalent to deciding whether T 
can be embedded into the 2-dimensional grid. This proves that 
solving the MEPL problem on the 2-dimensional grid NP-hard. 
To show NP-completeness it is again easy to see that deciding 
whether the MEPL is smaller than a given value problem can 
be achieved in NP. ■ 
Contrasting these two hardness results, the next sections 
will show that if one assumes a grid graph and a symmetric 
product distribution, nice algorithmic results can be obtained. 

V. Distributed MEPL with Symmetric Product 
Distributions 

For general request distribution it is hard to find a good 
or optimal solution even when one is not restricted to local 
and distributed switching rules. With this in mind, we first 
restrict ourselves to a simpler model of requests, namely, 
symmetric product distributions (Definition 3). Second, we 
assume d-dimensional grid topologies, and in particular the 
line and a 2-dimensional grid. We assume that a node learns 
the distribution from requests it is involved in and thus, it can 
decide whether the switching (exchanging positions) with a 
neighbor will increase or decrease the objective function, the 
expected path length of the network. The main result of this 
section is that under the above settings, a good approximation 
to the objective function can be found using only simple 
(greedy) local switching rules. To prove this result, we need 
the following definitions. 



A. Expected Distance to Center and Expected Rank 

To find a good placement for nodes which gives a good 
approximation to the MEPL, we define the expected center 
and the expected distance to it. 

Definition 4 {center and expected distance to the center). 

The expected center of a graph H and a symmetric product 
distribution V, is a node c*, s.t.: 

c* = argmin 2_] v{u)d{u,x). 

The expected distance to the center c* is: 

C = min > p(u)d(u,x), 



or equally: 



uev{H) 

C= J2 P(«)dKc*). 

uev{H) 



When H and V (represented by G) are clear in the context, 
both C and c* can be written as functions of a placement ip, 
i.e., C{lp) and c*{(p). The minimum expected distance to the 
center is defined then as Cmin = min^ C(iy?). The next lemma 
describes the relation between C and EPL. 

Lemma 3. For any given placement (fi, 2C{ip) > EPL(<^) > 

Proof: To see the upper bound, we suppose that instead 
of routing between two nodes directly, we route every request 
via the center c*. Routing a request in this way results in 
sampling two requests and summing up their distances to the 
center. In expectation, this is exactly 2C. Formally (for any 
^): 

EPL(<^)= V{u)v{v)-d{^{u),ip{v)) 

uveE{G) 

< PiuMv)id{^{u),c*)+dic*,^{v))) 

uveE(G) 

= E PWdlV'W^c*) + E PWd(c*,¥'('i^)) 

uev{G) vev{G) 
= 2C{v) 

The fact that C is a lower bound, we show as follows: 

EPL(vp)= Y Piu)viv)-diipiu),ipiv)) 

uveE{G) 

= E P(") E P(^)d(¥'(w),</'(^^)) 

uev(G) vev{G) 

> E P(") E P{v)d{c*,viv)) 
uev{G) vev{G) 

= Y p{v)d{c*,ip{v)) = C{^) 
vev(G) 



CoroUary 1. MEPL > C„ 



This follows since for the optimjil placement t^*: MEPL = 
EPL(^*) > Ci^*) > C^in. 

An important ingredient in bounding the performance of our 
local rules will be the following measure of expected rank. 
This quantity is an interesting measure on the concentration 
and uncertainty of a distribution. 

Definition 5 {Rank of nodes and the Expected rank). The 

rank of a node is the position of the node in the ordered 
list of nodes' probabilities (breaking ties arbitrarily). The 
node with the highest probability has rank 0. The rank of 
the node u ^ V is denoted as i{u). The expected rank 
of a probability distribution on the nodes of graph G is: 

E[i?]=E„ey(G)P(«M")- 

We next describe the local switching rules by which our 
distributed algorithm works. 

B. ( Greedy) Local Switching Strategies 

For two nodes u,v € V{G) and a placement cp, we say 
that M is a neighbor of v if and only if (p{u)ip{v) S E{H). 
A switching of u and v is taken to be understood as a new 
placement cp' where for each w G V(G),w u,v f'{w) = 
'^{w) and <^'(m) = <p(t') and '{>'{v) = <f{u), i.e., u and v 
switch places on H. 

We propose the following greedy strategy. A node switches 
with a neighbor if, according to the (observed) marginal 
distribution on the requests involving itself and its neighbor, 
switching positions improves the objective value. In this work, 
we consider two simple optimization rules: 

1) M-rule: Node will switch locations with its neighbor 
if the switch will minimize the objective function: the 
expected path-length between all pairs of nodes. This 
criterion is exactly the MEPL objective. 

2) C-rule: Node will switch location with its neighbors 
if the switch will minimize the expected path-length 
between the center node and all the other nodes. This 
objective does not give us a solution for the MEPL 
problem, but it will be used as an upper bound for it. 

If nodes switch only when this decreases the expected path- 
length (or some other criterion), then it is clear that this, strictly 
monotone, potential can not drop indefinitely (or too often) and 
thus, a (quick) convergence is guaranteed. A placement is 
said to be local minimum (or local optimum) if and only if 
no node in G can switch according to the rule they operate 
(i.e., M-rule or C-rule). When using the C-rule, we can prove 
the following about the local minimum placement. 

Lemma 4. Any local minimum placement Lp with respect to 
the C-rule, is center monotone.- If there is a path of switching 
directions from (p{u) to ip{v) that is distance-decreasing with 
respect to c* , i.e., a path for which every step is a switching 
direction and goes strictly closer to c* , then p(w) < p(w)- 

Proof Assume for sake of contradiction that lyj is a local 
minimum, that p{u) > p{v) and that there is a distance- 
decreasing path P of switching directions from (p{u) and 
to fiv). By induction there has to be two nodes u' and v' 



such that (p{u') and (p{v') are neighbors on the path P with 
p{u') > p{v') but v' is closer to c* then u' . By assumption it 
is possible to switch u' and v' and it is easy to see that this is 
an improvement with regards to the C-rule. This contradicts 
the assumption that is a local minimum. ■ 

Note that according to the C-rule, two neighbors switch 
locations only if the switch decreases C: the expected distance 
to the center. The improvement of the switch can be found 
locally, since the center location can be computed locally at 
each node via the expected position of its requests (which 
are identical to all nodes because of the product distribution). 
Therefore, the C-rule will greedily minimize, for each node, 
the distance to the expected position of its requests. 

Throughout the rest of this paper we assume that the 
system converges against a local minimum and analyze the 
performance of such a solution in this stable state. On the other 
hand, we do NOT assume anything about the starting position 
OR the specific order of the dynamics (node switches). Thus, 
in many cases, an initially random starting position converges 
(e.g., using random improving switches) to a (near) optimal 
solution; we make no such assumptions and assume a worst 
case sequence of improvements and a worst-case initialization. 

C. The Line - Linear placement 

First, we study a greedy local switch strategy on a 1- 
dimensional grid - the line. We assume that the C-rule switch- 
ing strategy is sequentially applied (in arbitrary order) on an 
arbitrary initial state and continuously adjust the network by 
switching neighbors. The strategy will converge against a local 
optimum from which no switch of two neighboring nodes 
improves the objective value in expectation. We are interested 
in quantifying how far such a locally optimal solution can 
be from the global optimum. The following theorem gives an 
answer for this question. 

Tlieorem 1. Let H be the line and V a symmetric product 

distribution, then any locally optimal solution achieved by the 
C-rule (or M-rule) is at most a factor of four larger than the 
global optimum of MEPL. 

We prove this theorem for the C-rule but this could be done 
similarly to the M-rule. Assume H and V as in the theorem. 
We first give an upper bound on the expected path length 
achieved by the C-rule in terms of the expected rank of the 
distribution (Definition 5). 

Lemma 5. For any locally optimal solution achieved by the 
C-rule on C{ip) < E[R] and EPL((^) is at most 2E[i?]. 

Proof Let d(iy9(v),c*) be the distance of ip(v) from c*. 
We want to bound it in terms of r„, the rank of v. From 
Lemma 4 we get that all nodes between ip{v) and c* on the 
line have higher probability than v and thus A{i.p{v), c*) < Vy. 

c{^)= v{v)d{^{v),c*)< Y'{v)v{v) =m] 

v£V(G) v(^V(G) 

From Lemma 3, EPL((^) < 2C((p) < 2E[i?]. ■ 



We now prove a lower bound for MEPL on the line and 
any symmetric product distribution of requests. 

Lemma 6. MEPL > C,„i„ > ^E[R]. 

Proof: Let ip* be the placement such that Cmin — C{lp*). 
Note that by definition, <p* minimizes the expected path length 
to the center Given the center c*{if*) and an arbitrary node v 
with a distance d((^*(w), c*) from c*, we want to find an upper 
bound on the rank of v by bounding how many nodes can 
have a larger activity level than v. Clearly, all such nodes will 
be at most at the distance d((y5* (w), c*) from the center, since 
otherwise, ip* will not be optimal. Since in a line there at most 
two nodes at distance i from the center d{Lp* {v),c*) > i(v)/2 
we obtain as desired: Cmin — X^ugy P('^)d('/'* (i^), c*) > 
E.p(«)r(«)/2 = ]E[i?/2]. ■ 
To conclude the proof of Theorem 1, we combine Lemma 3, 
Lemma 5 and Lemma 6 to get that for a local minimum ip: 

2E[R] > EPL{ip) > MEPL > ^E[R] (2) 

Thus, the ratio between the worst case local solution and the 
optimal solution is at most 4. 

D. The d-Dimensional Grid 

In this section, we extend the ideas from the line to grid net- 
works. Our results apply readily to grids of arbitrary dimension 
but, for sake of simplicity, we stick to two dimensions here. 
We first show that using the same greedy approach as in the 
line, namely switching neighboring nodes using the M-rule, 
leads to a drastically worse ratio between local and global 
minima. 

Lemma 7. On the d- dimensional grid, there is a local 
minimum with regards to the M-rule and tlie C-rule that is 
a factor of ^{n}/'^~^/'^ ) worse than the global minimum. 

Proof: We take ■n}/'^ nodes with uniform probability p = 
^-i/rf and set all other nodes to probability zero. We arrange 
the active nodes on a line along one dimension as an initial 
placement. We now note that this initial placement is a local 
minimum since all switches are non-improving with regards 
to the M-rule or C-rule and also lead to a higher expected path 
length. This is true because any switch between nodes in the 
line does not change the expected path length while switching 
a node from the line with an inactive node only increases the 
path length for the active node by one while the reduced path 
length for the inactive node has no effect on the expected 
path length. The same is true for the expected distance to the 
center. The expected path length of this linear arrangement is 
^i/d^2. To complete the proof we will now argue that there are 
arrangements with EPL of order 0(n^/'^ ). One such (close to 
optimal) solution can be achieved by arranging all active nodes 
within a ball around the center. Since the number of nodes in 
a ball in d dimensions grows as r'^ with the radius d it is clear 
that a radius of r — 0{n^^'^ ) suffices to contain enough spots 
to place the n^^'^ active nodes in. The maximum (and also 
expected) path length within the ball is also within a constant 
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Fig. 1. A local minimum when optimizing the distance to the expected 
position of a routing request (black node). The red nodes mark the positions 
that dominate the green node, that is, that have a path consisting of switch 
directions each going strictly closer to the black node. In a local minimum 
these nodes have to have a larger probability than the green node making the 
rank of the green node at least as large as the number of red nodes. 

factor of the radius, i.e., also 0{n^^'^'^). The ratio between the 
local minimum and the alternative ball arrangement is thus 
proving that the ratio between the worst local 
minimum compared to the global minimum is at least this 
large. ■ 
Note that the last lemma implies a Q{n^^'^) worst-case ratio 
for the 2-dimensional grid. Surprisingly, we can avoid this, 
locally stable but highly suboptimal solution, by allowing only 
slightly longer switches. The rule we propose is that a node 
can also switch with any of the neighbors in £i -distance three 
that differs two in one axis and one in the other (similar to 
the chess knight moves) and the switching is according to the 
C-rule. In this case we can prove the following. 

Theorem 2. Let H be the 2-dimensional grid and V a sym- 
metric product distribution, then any locally optimal solution 
achieved by the C-rule (and allowing "chess knight move" 
switches) is at most a factor of 4.62 larger than the global 
optimum of MEPL. 

The proof of this theorem is similar in spirit to the 1- 
dimensional grid, where we provide bounds on C for the 
optimal and locally optimal placements. First, we show that 
we get a fat set from this strategy and also prove a stronger 
rank-property of any locally optimal solution; namely, nodes 
that are far away from the center have to have a (quadratically) 
high rank. 

Lemma 8. For any local minimum of the C-rule C{ip) < 
-^E[\/R], where R is the rank of a given distribution (see 
Definition 5). 

Proof: We consider a local minimum of the C-rule 
achieved by cp. Without loss of generality we assume that 



is the identity function. Consider a node v in general 
relative position to the center c* (see Figure 1). We want to 
estimate how many nodes have a larger probabihty (higher 
rank) than the node v. To achieve this estimation, we analyze 
the area of the largest polygon, such that every node inside 
and on the edges of the polygon belongs to some distance- 
decreasing path from ?; to c*. According to Lemma 4 we then 
get the guarantee that for any local minimum with respect 
to the C-rule any node u in the polygon (with a distance 
decreasing path from v) has p(v) < p(u). Figure 1 provides 
an example for this: The node xi is closer to the center than 
V and switching between xi and v is possible. This impUes 
that p{v) < p{x\). Furthermore, if we look at the distance- 
decreasing paths v,xx,X2t ■ ■ ^ X'j and w, zi, Z2, • • ■ , a^r we ob- 
tain from Lemma 4 that p(w) < p(a;i) < p{x2) < ■ ■ ■ < pixr) 
and that p{v) < p(zi) < . . . < pixj). These paths can 
furthermore be extended to any node in the polygon. All these 
nodes have such higher activity levels then p{v). 

To get a bound on the rank of v, i.e., in the number of 
nodes that have a larger activity level, we count the number of 
nodes Stotai that are inside the polygon. This number generally 
involves many floors and ceilings. We avoid these by first 
calculating the number of nodes A{x) bounded by a right 
triangle shape that starts at a point and whose (axis parallel) 
legs have length x and x/2. We denote this number by A{x) 
and in the following give a formula and an estimate for it that 
holds for any positive real x: 



A{x) :- 



lx/2\ 

E 

i=0 



{[x\+l-2i) > 



+ 1 



if X > 1, 
if a; < 1. 



Now we are ready to calculate Stoiai- We denote the number 
of nodes in the middle rectangle as ^i, the number of nodes 
in the upper triangle as ^2 and so on according to Figure 1. 



Si = {a - l){b - 1) 
S4, = A{a + b/2) 



52 = A(a-l) S3 
S5 = A{b - 1) 



A{h + a/2) 



So, we obtain that Stotai = 5*1 + 5*2 + S'3 + ^4 + S'5 — 3, where 
—3 is needed since the node v should not be calculated (but 
was counted twice) and c* should be calculated once (but was 
counted twice). By adding up the expressions we obtain for 

a,6 > 2: 

Stotai = /Si -h ^2 -h S's + 'S'4 + 5*5 — 3 

>(a-l)(6-l) + ^^^+ 



(6+1)2 , (a +1)2 



4 

(6-1)2 



+ 



+ 1> ^(a + 6)2 



Easy to verify that the inequality Stotai > ^(a + 6)^ holds 
also in the case where a, or h, or both equal to 1. Since the 
rank of v is at least StotaU we get r(v) > j^{d{v,c*)Y, and 



thus: d(w,c*) < ^^Jv{v). Now, 



4 E[Vi?]. 



Next we prove a lower bound for the cost of the optimum 
placement obtaining a similar expression in terms of the 
(expected) rank. 

Lemma 9. MEPL > C^in > 

Proof: Let 93* be the placement such that Cmin = Ci^p*)- 
Note that by definition (p* minimize the expected path to the 
center. Given the center c*(<^*) and an arbitrary node v with 
a distance d{v.c*) from c*, we again find an upper bound on 
the rank of v by showing how many nodes can have a larger 
activity level than v. Again, all such nodes will be at most 
at the distance d(u,c*) from the center, since otherwise the 
solution will not be a global optimum. There are exactly 4 
nodes at distance 1 from c*, 8 nodes at the distance 2 and in 
general 4i nodes at distance i. This lead to the to the following 
bound: 



d(u,c*) 

r(t;) < 4i : 



2{d{v,c*)f 



So, we obtain d(v, c*) > \/t:{v)/2, and thus: 



C = ^ V{v)d{v, c*)^Y. V{v)d{v, c*) 
v€V vev 



Now we are ready to prove the result of Theorem 2. From 
Lemma 3 and 9 we get that the minimum expected path length 
is at least ¥,[^/R/2]. From Lennma 8 we obtain that C{ip) < 
E[^V^], and thus by Lemma 3 the expected path length of 
a local minimum can not be larger than 2E[^^/R]. Therefore 
the ratio between the optimal solution and any local minimum 
witii regards to the C-rule is at most 2^ w 4.62. 

VL Clustered Requests 

We have demonstrated that self-adjusting networks and 
their local switching rules work well on grid networks with 
symmetric product distributions. We now briefly discuss some 
interesting preliminary results on a more general type of 
request distributions: clustered requests. For this we consider 
situations where processes can be clustered into groups such 
that communication takes place only or predominantly be- 
tween processes belonging to the same group. This locality 
is inspired by practice and we beheve that such a structure in 
the requests is quite common. 

Ideally, a self-adjusting network "detects" such clusters and 
arranges processes such that groups will reside in separate 
parts of the network infrastructure. Such an arrangement 
facilitates short routes since requests between group members 



get routed quickly without leaving the group. Such a well 
clustered placement of nodes can be a drastic improvement 
of the expected path length compared to a non-optimized 
placement. In particular, any placement that is oblivious to 
the clustering, e.g., a random placement, will have a bad 
performance - leaving plenty of room for improvement. We 
believe that the simple switching strategies presented in this 
paper perform very well in many such settings. 

Our investigations and simulations on d-dimensional torus 
topologies (see Figure 3) have led to several interesting pre- 
liminary results in this direction: On the negative side we were 
able to construct local minima in many topologies that have 
a poor performance compared to the global optimum. This 
shows that it is not possible to give the same type of strong 
approximation guarantee independent of the initialization. In 
simulations we observed that for d = 1, i.e., on a ring 
topology, even random initializations lead to bad local minima. 
The reason for this is that connectivities in the ring are 
too restricted to allow the resolution of distributed clusters 
without disturbing other, already fixed, clusters. Fortunately, 
this changes drastically for higher dimensions. For d > 1 
local minima become extremely unstable and sensitive to 
small perturbations. They can only occur if one starts in 
a carefully constructed worst-case initialization. This obser- 
vation is supported by our simulations which produce very 
promising results. Figure 3 shows the outcome of some of 
these simulations in a 2-dimensional torus network. We plan 
to investigate the performance of our self-adjusting networks 
on this and other topologies in future work. 

A. Simulation Results for 2-D Torus 

In this section we show simulation results for a 2- 
dimensional torus topology. We assume a uniform clustered 
distribution of requests with uniform clusters' activities. 

Figure 3 shows an example for the improvement in a 2- 
dimensional torus. Overall, the greedy switching algorithm 
successfully detects clusters and groups them together How- 
ever, there are still some clusters that are not connected. In 
Figure 3 (a) we can see the initial random placement of the 
clustered nodes. Nodes with a black bold frame are the centers 
of their clusters (as was defined earlier, a center is a node that 
has the minimal expected distance to all other nodes in the 
cluster). In Figure 3 (b) we see the placement of the nodes 
achieved by the greedy M-rule switching strategy. Although it 
looks that the nodes are highly grouped, we can see that the 
shapes of the clusters are not optimal (an optimal placement 
should look like a circle around the center of the cluster). 
Some clusters are stretched (e.g., brown cluster) and some are 
even not connected (e.g, orange cluster). 

When every node on the torus belongs to an active cluster, 
we can frequently run into situation in which two nodes will 
not switch even if it is improvement for one of the clusters. 
This suboptimal local solution can be improved if we allow 
some nodes on a torus to be inactive. In the following two 
figures we see the results of such simulation where 50% on 
the nodes are inactive. In Figure 3 (c) we can see the initial 
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Fig. 2. Greedy switching on Ring and 2-dimensional torus. Ratio between 
the initial and the final EPL, as a function of number of clusters. 



random placement of the clustered nodes, (the inactive nodes 
are white colored). In Figure 3 (d) we see the placement of a 
local minimum of EPL achieved by the same greedy switching 
strategy. But now we can observe much nicer concentration 
of the nodes around their centers. These figures lead to many 
interesting future research questions about the topic. At the 
Figure 3 (c) there is a QR link to the animated version of the 
figures. 

B. Bad Local Minima for Ring Graph 

We first give a formal model for the clustered requests 
distribution. We assume, for simplicity, that the clustered 
requests distribution is a mixture of product distributions on 
disjoint supports. In the case that the product distributions are 
uniform, this results in the following. Let k be the number 
of groups and let Vi , . . . , 14 be the vertex disjoint node sets 
of these groups and let Vq ^ V \ [J^Vi be the set of inactive 
nodes. Let for 1 < i < /c be the activity level of each group 
such that X^iLi Pi ~ ^- Given this we have that the probability 
p(u, v) for a request between a nodes u,v &V is: 







u,v € Vi , 
otherwise 



> 



(3) 



For the 1-D ring graph, the performance of the greedy 
switching strategy is bad. First, the simulation results pre- 
sented in Figure 2, show the improvement ratio between the 
EPL of a random starting locations to the EPL of a local 
minimum reached by the local greedy M-rule as a function 
of number of clusters. In the ring graph, the improvement is 
almost negligible and it is constant with the number of clusters. 
Notice, when a greedy switching strategy is applied to a 2- 
dimensional torus graph, the improvement ratio is much better 
and it is growing with the number of clusters. 

Next, we prove the following lemma that shows that the 
ratio between the EPL of a worst case local minima and the 
EPL of the best placement is not a constant anymore but is 
growing with number of clusters. 



(a) - Initial random placement. | Vb | = 0. 



(c) - Initial random placement. |Vb| ^ n/2. 



(e) 




Final placement 




(d) - Final placement 



Fig. 3. (a) Torus with 900 nodes, 16 clusters and no nodes in the inactive cluster. Nodes are placed at random positions, (b) Final placement after applying 
the local greedy strategy in a round robin fashion. All clusters are grouped together, but their shapes are not optimal for a given cluster, (c) Torus with 900 
nodes, 8 active clusters and half of the nodes in the inactive cluster. Vc) has n/2 nodes. Nodes are placed at random positions, (d) Final placement after 
applying the local greedy strategy in a round robin fashion. All clusters are grouped together and almost optimally shaped around their center, (e) Link to the 
animation of this simulation (http://www.bgu. ac.il/~avin/pmwiki/pmwiki.php?n=Main.Self-AdjustingNetworks). 



Lemma 10. In a ring network with uniform clustered distribu- 
tion of requests, the ratio between the worst case stable nodes 
placement (i.e., a local minima) and the best case placement 
is at least 1.5c, where c is the number of clusters. 

Proof First we show that there exists a bad local min- 
imum on the ring graph and uniform clustered distribution 
of requests. We assume that there are c clusters of 2fc + 1 
nodes each, and thus the total number of nodes in a ring is 
n = c(2/c+l). Let Lpi be the following placement of the nodes: 
{v\,v\, vf,. . . , V2,V2, V2, ■ ■ ■), where vf £Vj. We now prove 
that the placement of nodes (pi is locally optimal, i.e., there 
are no improvement switches. 

Let's denote the expected cost of the paths where the node 



V is either source or destination as E„. Due to the symmetric 
location and the uniform distribution, E„ is the same for all 
the nodes v G V. Let's assume that v E Vi, where Vi is the 
set of nodes in cluster 1, and |Vi| = 2k + 1. Let's denote the 
probability of choosing the node u e Vi as a pair for the given 
node V gVi as Clearly, — y^^. Before the switch: 



^ ^ Pu\v^vu 

ueVi 



1 



C • I 



1=1 



c-k{k + l) 
2k + 1 



Due to the symmetry, there is no difference with what node 
we will try to switch the node v. So, let's assume that we 
switch it with its right-hand neighbor - w. In order to check 
the difference with the path cost before the switch, we need 



to consider all the paths in which nodes v and w are involved. 
Clearly, even after the switch E„ = due to the symmetry, 
so we will now find the E,, after the switch and will compare 
it to the E^ before the switch. After the switch: 



■j.eVi 



fc(fc + 1) 

2fc + 1 



Since all possible switches are the same due to the symmetry, 
we can conclude that there is no switch that will improve the 
expected paths cost and the placement (pi is a local minima. 

Now let's assume different placement of the nodes - cpopt- 
{vl,vl,...,vly^^,vl,v^,...,v'^y^^,...), where vj G Vj. For 
this, probably optimal, placement, we wiU calculate now the 
expected path length (EPL). 

We consider the part of the ring of 2fc+ 1 nodes that belongs 
to a single cluster. Number of paths of length 1 inside the 
cluster is 2 • 2k. Number of paths of length 2 is 2 • (2fc — 1), 
length 3 is 2 • (2A; - 2), length m is 2 • (2fc - m + 1), and of 
length 2k is 2. So, we obtain the average path length in one 
cluster: 



IE— ^ Pvudvu ~ 2 



2k 



2^m{2k-m + l) 



A;(2A: + l)(4fc + 4) 
3 -2(2^ + 1)2 



i=l 

k{2k + 2) 
3(2fc + l)' 



VII. Conclusions and Future Work 

In this preliminary work, we formally defined the MEPL 
problem which has practical significance in saving energy 
of fixed infrastructure network. We showed that in general 
cases, the problem is hard to compute, but under some realistic 
assumptions on network infrastructure and traffic patterns, we 
propose efficient local and distributed algorithms that achieve 
almost optimal solution. Our algorithms are based on migra- 
tion of processes, which allows network optimization without 
changing the underlying infrastructure. This idea integrates 
well with an SDN concept which will probably include process 
migration functionality in its management platform. 

In future work, we plan to extend our results to other 
topologies that are used in data centers networks, e.g., fat 
trees [28]. We also aim to investigate other types of requests 
distributions that are based on real data. 
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